Yes its very frustrating and they sit proud without the black clips. You can find some old clips but 99% of the time they break when you try to take them off. We ended up grinding down the roof plugs a bit and it improved it but still did not sit right.
I have replaced the roof trims twice and still not happy :(
Can't really help other than to let you know you are not going crazy.